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TRODUCTION 


We are at a pivotal time in efforts to improve 
the health, education, and well-being of 
children, youth, and families in the United 
States. There is increasing recognition that 
population-level results cannot be achieved 
without closing persistent equity gaps—gaps 
defined by race, class, income, ethnicity, 
immigration status, gender, and urban/rural 
geography—that are severe and changing 
slowly, if at all. Further, we know equity gaps 
won't close without recognizing and acting 
on the fact that people of color are too often 
trapped in multi-generational poverty and in 
neighborhoods with tough living conditions 
because of structural factors that stretch 

far back in our nation's history, including 
genocide, enslavement, institutional racism, 
anda history of social exclusion. 


The encouraging news is that leaders in 
philanthropy, in government, and within 
many communities are implementing 
strategies to address health disparities, 
reduce poverty and boost incomes, create 
effective schools, and transform tough 
neighborhoods—all to achieve better, mor 
equitable outcomes. These complex initiatives 
usually involve transforming public systems 
or revitalizing communities or both. In our 
view, they require equally transformative 
approaches to generating and applying 
knowledge and evidence. Such approaches 
require the following elements! 


Ensuring that those most affected 
by change—community residents, 
those involved with public systems, 
others—participate in generating 
and applying evidence. 


Using all available sources of credible 
knowledge to understand the problem to 
be solved, the context within which it is to 
be addressed, and to design the solution. 
Maintaining a steadfast focus on results. 
Deploying multiple methods of evaluation 
to gauge the progress of complex 
initiatives, which often involve policy and 
systems change and shifts in dominant 
cultural narratives over many years. 
Harvesting learning and evidence from 
on-going innovation in ways that promote 
continuous improvement and adaptation 
of what has worked in the past. 


We need to lift up, learn from, refine, and 
invest in generating and applying knowledge 
and evidence to proactively contribute to 

and accelerate the achievement of more 
equitable outcomes. 


In this paper, we turn our attention to 
considering these elements in light of what it 
means to “place equity concerns at the center” 
of improving outcomes for those who are the 
most marginalized. Our premise is that how 
vidence is generated and applied—ie., the 
assumptions made and methods chosen for 
this task—can affect how rapidly we'll learn and 
how effectively we'll make progress in achieving 
more equitable outcomes.? We need to lift up, 
learn from, refine, and invest in generating 
and applying knowledge and evidence to 
proactively contribute to and accelerate the 
achievement of more equitable outcomes. 


1 Schorr, Lisbeth and Frank Farrow. “An Evidence Framework to Improve Results.” Center for the Study of Social Policy, 2014. Available at: https://odfs.semantic- 


scholar.org/Ofcl/O2cb2966e2959ebdc691c2936565cd6d3e99. pdf. 


2 Bryk, Anthony. “Redressing Inequities: An Aspiration in Search of a Method.” Carnegie Foundation for the Advancement of Teaching, April 2017. Available at: 


https://www.carnegiefoundation.org/resources/publications/redressing-inequities-an-aspiration-in-search-of-a-method/. 


PUTTING EQUITY at the CENTER 


What does it mean to put equity at the center 
of evidence development and use? It means 
that equity is not a by-product but an essential 
lement—a value—of thoughtfully considered 
intervention design, learning agendas, and 
applied data collection and evaluation and 
research. This means that inequities and 
disparities are the problems to be solved, not 
simply documented. Anthony Bryk and his 
colleagues at the Carnegie Foundation for 
the Advancement of Teaching advocate for 
“new ways of thinking about and acting on 
educational inequities...oy explicitly focusing 
on disparities as the problem to be solved... 
investigating the root causes...in the lived 
experience of all involved...exploring Now...our 
work may act to sustain the disparities that 
trouble us so greatly.”* Similarly, in another 
hemisphere, the New Zealand Ministry of 
Education has focused the use of evidenc 
over the past two decades to support 
“educational improvement in valued 
outcomes for diverse (all) learners with a 
priority for accelerated improvement for 
learners who have been underserved in 
their education or disadvantaged.”* 


Philanthropic leaders as well as evaluators and 
researchers are giving increasing attention to 
making equity central to evaluation. In 201), 
the American Evaluation Association (AEA) 
issued a statement on cultural competence 

in evaluation that urged development and 
implementation of evaluations in a context 


3 Ibid. 


of social inclusion and equity.° While this 

is an important step, more will be needed: 
“inclusion” does not in itself mean change 

to the systems of power and privilege that 
too often influence decisions about evidence 
generation and use. It also does not mean 
examining underlying assumptions in the 
methods and orthodoxies traditionally 
applied in evaluations and research. In efforts 
to push the concept of equity in evaluation 
further, some Native American researchers 
have focused on designing an indigenous 
evaluation framework? and others have been 
refining participatory research methods as 
steps to ensure that people most affected 

by interventions are part of research design 
and implementation. Most recently, several 
foundations are supporting use of “Equitable 
Evaluation,” a framework that integrates 
cultural competence and equity.’ 


“Ethnic and cultural minority communities 
are too often left behind in efforts to build 
evidence to inform effective interventions.” 
— Nancy Rumbaugh Whitesell, Center for American 


Indian and Alaska Native Health 
OPRE Report #2017-76 


These and other changes acknowledge that 
there is a serious “evidence gap”.® We simply 
do not have sufficient, credible evidence about 
what works and why for different populations. 


4 Alton-Lee, Adrienne. “The Use of Evidence to Improve Education and Serve the Public Good.” American Educational Research Association April 2012. 
Available at: https:/Wwww.emeraldinsight.com/doi/abs/10.1108/S1479-368720150000022017. 


5 Dean-Coffey, Jara et al “Raising the Bar—Integrating Cultural Competence and Equity: Equitable Evaluation.” The Foundation Review 6: no. 2, 2014. Available 


at: https://doi.org/10.9707/1944-5660.1203. 


6 La France, Joan and Richard Nichols. “Reframing Evaluation: Defining an Indigenous Evaluation Framework.” The Canadian Journal of Program Evaluation 23, 


no 2, pp. 13-31, 2010. Available at: https://eric.ed.gov/?id=EI935344, 


7 Luminare Group et al. “Equitable Evaluation Framing Paper Equitable Evaluation Project, 2017.” Available at: https://drive.google.com/file/d/IRBWoWVduM- 


vh4S68qgdmPLN3WPEyPWOQwOS/view. 


® Whitesell, Nancy Rumbaugh. “Evidence and equity: Challenges for research design.” Colorado School of Public Health, 2017. Available at: https:/Awww.acf.hhs. 


gov/sites/default/files/opre/whitesell_opre_brief_clean_508.pdf. 


PUTTING EQUITY at the CENTER 


For example, among “Twenty home visiting 
models with evidence of effectiveness; only 
one...has shown effectiveness data specific 
to cultural groups.’ And, as one researcher 
has noted, “if the trajectory of evidence 
continues to build for populations easiest to 
engage and study while continuing to lag for 
marginalized or hard-to-reach communities, 
equity gaps are likely to widen.”"° 


“If you don’t mention my race, you render me 


mute. And | might then render your analysis 
insufficient.” 


— Shiree Teng, Consultant 


An example of an evidence gap exacerbating 
existing inequity comes from the elementary 
math classes of New Zealand. Concerned with 
declining student math scores, the Ministry 
of Education embarked on a professional 
development program intended to improve 
teacher proficiency. It also included grouping 
students by abilities (or teacher perceived 


abilities) referred to as “streaming.” The aim was 


to have all students be “literate and numerate.” 
Test scores did improve, but not for Maori and 


to design and study a “culturally responsive” 
teaching model within multi-ability groups." 
The new approach is showing promise but is 


also challenging teachers to understand cultural 


context and apply it in their teaching as well as 
moving away from streaming which tended to 
encourage competitiveness in contrast to the 


“Pasifika notions of the value of collectiveness.” 


Pasifika students. In fact, their scores continued 
to decline. Out of this disappointing result grew 
an effort by indigenous New Zealand researchers 


In short, if we want (1) greater innovation 

in addressing inequities, and (2) a sturdy 
body of evidence from both research and 
experience to guide our efforts to combat 
inequities, we have to check our assumptions 
about methods and rigor and adapt our 
approaches to generating, gathering, and 
applying evidence, especially as we track 
progress of community transformation 

and systems change initiatives. We have th 
opportunity to do this—and enough examples 
of where at least pieces of this are being 
done—that we can indeed put equity at 
the center of our approach to evidence. 


To promote an equitable approach to 
generating and using knowledge, we believ 
three essential elements must be included in 
the knowledge development enterprise: 


1. Valuing and using the perspectives and 
knowledge of people most affected by 
the root causes of inequity and by the 
proposed solutions. 


2. Understanding and articulating the 
structural, systemic, cultural, and historical 


factors—slavery, systemic oppression, white 


supremacy, and privilege—that are root 
causes of inequities. 


3. Focusing on the variations in impact 
of interventions—not just “what works” 
on average, for some individuals, 
somewhere—to discern what adaptations 
are needed to reduce disparities. 


° Ibid. 
1 Ibid. 


" Hunter, Roberta et al. “Learning by Leading: Dynamic Mentoring to Support Culturally Responsive Mathematical Inquiry Communities Mathematics 
Education Research Group of Australasia, 2016. pp 59-73. Available at: https://eric. 


Bills, Trevor and Roberta Hunter. “The Role of Cultural Capital in Creating Equity for Pasifika Learners in Mathematics” Proceedings of the 38th Conference of 


the Mathematics Education Research Group of Australasia, 2015.pp. 109-116. Available at: https://files.eric.ed.gov/fulltext/ED572423.pdf.ed.gov/?id=ED572401. 
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ALUING and using the PERSPECTIVES 
AND KNOWLEDGE OF PEOPLE MOST 


AFFECTED BY INTERVENTIONS 


Many researchers and evaluators of ambitious 
social change efforts, as well as participants 

in the activities they sponsor, recognize the 
importance of engaging the people most 
affected by change strategies to own the 
strategies and the learning and knowledge 
development that accompanies them. 

This conviction is value-based. Principles of 
fairness, democracy, and justice require voice 
and agency by those most affected. Beyond 
being the right thing to do, it is practical and 
productive. Community ownership produces 
richer, more reliable findings.” Conversely, 
when the lived experience of those most affected 
by change is invisible to those trying to solve 
a problem or develop evidence about it, 

the strategies conceived and the evidence 
generated are likely to be flawed, and change 
efforts risk repeating patterns of colonization 
and evidence extraction. 


While recognized conceptually, the 

practice of assuring that people most 

affected by interventions are part of design, 
implementation, and evaluation/evidence 
development is rarely done, at least in a 
comprehensive way. Too often, good intentions 
and good beginnings that involve community 
residents in all these areas fall by the wayside 
as decision makers realize what a profound 
shift is required to share and elevate power 

of those historically without power. 


“the key idea remains: as you focus in on trying 
to address an educational inequity, bring the 
voices of the people who are most directly 
impacted into the conversation. Seek to 
understand the dynamics of disparity through 
the eyes, mindsets, and emotions of all involved.” 


— Anthony Bryk, Carnegie Foundation for the 
Advancement of Teaching 


Time and resources are frequently cited 
barriers to developing the necessary reciprocal 
trust with community members around 
everything from defining the questions to be 
addressed, the data to be collected, the typ 
of findings that will be produced, and how 
they will be used. But these are barriers of 
our own making, reflecting what is currently 
most valued in the worlds of research and 
evaluation. As Nancy Rumbaugh Whitesell 
has noted, with regard to community-based 
participatory research, “Researchers often 
understandably shy away from doing this 
kind of research. Academic pressures for 
publication and promotion are often at 
odds with community pressures for genuin 
engagement, local dissemination, and creative 
thinking about the meaning of evidence.”® This 
is exactly why putting equity at the center of 
research, evaluation and evidence building is 
critical. The later we start, the more behind w 
will be as a field. There is urgency to do this now. 


® Luchenski, Serena et al. “What works in inclusion health: overview of effective and interventions for marginalized excluded populations” The Lancet 391, no. 
10117, January 20, 2018. Available at: http://dx.doi.org/10.1016/SO140-6736(17)31959-1. 


4 LaFrance. Joan. “Culturally Competent Evaluation in Indian Country.” New Directions for Evaluation2004. No.102, summer 2004. pp. 39-50. 


VALUING and using the PERSPECTIVES AND KNOWLEDGE 
OF PEOPLE MOST AFFECTED BY INTERVENTIONS 


Working directly with the people most affected 
by research actually enhances research rigor 
and the validity of findings. It increases the 
likelihood that findings will be translated 

into sustainable solutions and practices. 

To quote Whitesell again, “Close partnerships 
with communities contribute to rigorous 
intervention science, beginning with the 
collaborative identification of the problem to 
be addressed, through the selection or creation 
of an intervention, implementation, evaluation, 
dissemination, and sustainability plans. 
Sustained partnerships between communities 


knowledge development simply do not fit 

with the requirements of genuine research 
partnerships. The way research is carried 

out, funded, and the timeframes associated 
with it are incompatible with what it takes 

to generate, discover, and apply the needed 
evidence from the complex and evolving 
interventions that are most promising. Also, the 
predominant research/evaluation conventions 
often make little sense to those whose lives and 
communities are the most involved in change 
strategies. This has to change if we are to us 


and researchers can improve the relevance 

of questions, appropriateness of design, 

and feasibility of implementation and data 
collection. Together, partners bring state-of- 
the-art scientific methods and deep knowledge 
of the local community and culture to prevent 
problems and promote health equity.” 


Evidence gathering aimed at advancing equity 
is more than including community members 
on research teams or interacting more directly 
with the subjects of research or evaluations. 
Those are important steps as long as they 

are seen in a broader commitment to share 
power as part of the research and/or evaluation 
endeavor. That broader commitment 

can then be reflected in the ways that 
research, evaluations, and other knowledge 
development enterprises are budgeted and 
staffed, i.e., building in the resources so that 
participation by people most affected by the 
research is a fundamental premise. 


But while such partnerships are increasingly 
recognized for their important contributions, 
it is also clear that existing paradigms for 


vidence to advance more equitable outcomes. 


tis a change that is advocated for by researchers 
from many nations who are concerned with 
research conducted in indigenous communities. 
Examples of this advocacy include: 


In the US, researchers have suggested, 
“Including community members in data 
analysis and interpretation may not be 
a natural process for most researchers. 
With tribal communities, however, it is 
important that data be analyzed within 
the context of the local community.’"” 

- Also inthe US, authors of the Indigenous 
Evaluation Framework (IEF) hav 

noted, “IEF involves a fundamental 
paradigm shift in which the nature 

of knowledge itself expands beyond 
empirical knowledge to include 
traditional knowledge and revealed 
knowledge. It redefines culturally bound 
understandings of “actionable evidence” 
and privileges place-based, experiential 
knowledge as valuable to learning and 
improving both programs and the broader 
communities of which they are a part.”"® 


8 Whitesell, Nancy Rumbaugh. “Evidence and equity: Challenges for research design.” Colorado School of Public Health, 2017. Available at: httos://www.acf.bhs. 


gov/sites/default/files/opre/whitesell_opre_brief_clean_508.pdf. 
% Ibid, p.4. 


VALUING and using the PERSPECTIVES AND KNOWLEDGE 
OF PEOPLE MOST AFFECTED BY INTERVENTIONS 


In Canada, a Métis tribe researcher 

has explained “..that the foundation 

of research is the lived indigenous 
experience and this must ground the 
work, not theories or ideas that are 
brought to bear on this experience. 
Theories will soring from the people 
themselves—theories that explain the 
many facets and connections of our 
individual and collective lives." 

South African researchers have noted, 
“It is incumbent on us as researchers that 
we do not take community-generated 
data and wisdom and walk away. 
Through consultations with participants, 
culturally relevant processes, resources, 
and knowledge outcomes can be shared 
in ways that benefit the community.’2° 


Responding to these challenges 

requires evaluators, firms, and academic 
departments to make seismic shifts in 
the processes and practices of knowledge 
development to bring the level of needed 
balance to these power relationships. 


Changing knowledge development practices 
ina way that assures the type of partnership 
described by these international and domestic 
examples is critical. The toughest issues 
involved in creating these new partnerships 


are often about traditionally white-led and 
white dominated research efforts sharing 

or relinquishing power and control to 
communities. Similar challenges arise around 
academic-led research efforts. Responding 

to these challenges requires evaluators, 

firms, and academic departments to make 
seismic shifts in the processes and practices of 
knowledge development to bring the level of 
needed balance to these power relationships. 


Such seismic shifts require a fundamental 
commitment and value and therefore an 
emphasis on “sharing information, decision- 
making, power, resources, and support 
among members of the partnership.””' This 
means that, “Researchers must be willing to 
put themselves in the position of becoming 
the learner, accepting new viewpoints, and 
giving up their position as the sole expert 
in the partnership.”2? However, even this 
observation still falls short of advancing 
equity in knowledge development. As long as 
researchers are paid to “become the learner” 
but community members are not resourced 
to be the teachers, power is not fully shared. 
Both researchers and community members 
must be adequately resourced and respected 
so that both can play their dual roles of 
teaching and learning. 


” LaVeaux, Deborah and Suzanne Christopher. “Contextualizing CBPR: Key Principles of CBPR meet the Indigenous research context.” Pimatisiwin, 2006. 


Available at https:/Avww.ncbi.nim.nih.gov/pmc/articles/PMC2818123/. 


8 La France, Joan et al. “Culture writes the script: On the centrality of context in indigenous evaluation.” New Directions for Evaluation 2012, no. 135, September 


2012. pp. 59-74. 
® Ibid, pp 64. 


20 Keane, Moyra et al. (2017). “Decolonising Methodology: Who Benefits From Indigenous Knowledge Research?” Educational Research for Social Change 6no. 1, 


2017. pp. 12-24. 


21 LaVeaux, Deborah and Suzanne Christopher. “Contextualizing CBPR: Key Principles of CBPR meet the Indigenous research context.” Pimatisiwin, 2006. 


Available at https://www.ncbi.nilm.nih.gov/pmc/articles/PMC2818123/, 
2 Ibid. 
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UNDERSTANDING //7e STRUCTU RAL, 


SYSTEMIC, CULTURAL, and HISTORICAL 
FACTORS that ave ROOT CAUSES OF INEQUITIES 


Listening to and valuing the lived experienc 

of people who have been marginalized and 
oppressed begins to bring the structural 
impediments to equity into focus. Most 

chronic equity gaps are linked to larger structural 
factors that change slowly, if at all, in many high 
poverty communities: historical oppression 

and on-going discrimination; criminalization of 
populations of color; and income and wealth 
gaps that are rooted in historical privilege of 
white people. Yet, knowledge development 

has often focused primarily on “down-stream" 
interventions—those that address individual 
behaviors and risk factors rather than these 
persistent structural obstacles.* Fortunately, 
public health research and strategies are 
increasingly looking “upstream” to better 
understand the social determinants of health 

in communities. One study team has noted, 
“Increasing attention is now being given to policy 
research focusing on the built environment and 
the food environment in Latino communities 
with the assumption that such information 
would help mitigate ‘top down’ decisions.” 


Examining the systemic and structural roots 
of inequity, while complex, is possible, and the 
knowledge gained is essential for designing, 
implementing, and assessing initiatives that 
strive to achieve equity at the scale needed to 
be truly transformative. Examples of this kind 
of evidence building include the following: 


“It starts with...seeing how our educational 


systems actually create the unsatisfactory 
outcomes we observe.” 


— Anthony Bryk, Carnegie Foundation for 
the Advancement of Teaching 


Institutional ethnographic studies 
produce accounts of how institutional 
practices result in problematic 
outcomes for children, youth, and 
families.*° In conducting an Institutional 
Analysis to generate information and 
evidence for addressing disparities in 
outcomes for children of color involved 
in child welfare, CSSP has found that 
“Youth of color are too often placed 

in communities that are distant from 
providers offering required services, 
visitation sites, and court hearings. 
Historical inequities in transportation 
patterns and how buses are routed 
through communities can disparately 
affect the ability of youth of color and 
their families to travel to and participate 
in visitation and other mandated 
services in a timely manner. 
Sometimes child welfare systems 
mandate that youth attend programs 
such as substance abuse, parenting, 

or domestic violence programs with 
providers that are not accessible to 


23 Marmot, Michael. “Inclusion Health: Addressing the Causes of Causes.” The Lancet 39], no. 101117, January 20 2018. Pp. 186-188. Available at: http://dx.doi. 


org/10.1016/S0140-6736(17)32848-9. 


4 Gonzalez, E.R. and Villanueva, S. “Communities Creating Healthy Environments to Combat Obesity: Preliminary Evaluation Findings from Two Case Studies.” 


Californian Journal of Health Promotion 10, Special Issue: Health Disparities in Latino Communities, 88-98, p 87 


25 Weber, Kristen and Sarah Morrison. “The Institutional Analysis: Uncovering Pathways to Improving Public Systems and Interventions for Children and Fami- 


lies.” The Center for the Study of Social Policy, 2015. Available at: https://cssp.org/resource/ia-methodology-web/ 


UNDERSTANDING #/¢ STRUCTURAL, SYSTEMIC, CULTURAL, 
and HISTORICAL FACTORS ¢/at are ROOT CAUSES OF INEQUITIES 


communities of color or not culturally 
relevant. Often these services fail 
to address the underlying trauma 
experienced by youth of color and 
do not support them in the healthy 
develooment of their racial/ethnic 
identity. Systems of accountability rarely 
look at questions of service accessibility 
and cultural relevance.”26 

- Measures of such intangible forces 
as who holds power, and how it 
is wielded, are being developed.?’ 
Researchers at the University of 
Southern California evaluating efforts of 
The California Endowment have noted, 
“Policy successes should be measured 
not only in terms of what polices are 
proposed, but also how different types 
of power have been built. This includes 
how the community's understanding of 
issues or structures has deepened, how 
leadership has been nurtured, and how 
community involvement has grown."® 


Some change initiatives recognize these 
broader structural forces and deploy 
strategies to affect these forces directly 

or indirectly. For example, The Robert 
Wood Johnson Foundation (RWJIF) and 
The California Endowment (TCE)—two 
foundations advancing health equity—are 
using approaches based and influenced 
by a recognition of the social determinants 
of health framework, a research-based 
framework that identifies the larger factors 
that have been shown to affect population 


health more significantly than the quality 


of health care itself or individual behavior.”? 
Figure 2 illustrates how a social justice 
lens rooted in the social determinants 
influences health outcomes in an RWI 
supported initiative, Communities 
Creating Healthy Environments. 


FIGURE 1 
USING A SOCIAL JUSTICE LENS 
to IMPROVE HEALTH EQUITY 


Social justice refers to the understanding that all people 
should be treated fairly, have equal access to goods 

and resources, and have the right to self-determination 
and cultural expression. This lens prompts program 
developers to ground their interventions in the particular 
social justice perspective of the grassroots organization 
spearheading health promotion. 


An organization possessing a place and infrastructure 
justice perspective...may view a targeted health problem 
as a consequence of the lack of safe recreational 

spaces for physical activity. Such a lack may result 

from limited media, political, or economic attention 

to the recreational and health needs of community 
esidents. Consequently, health promotion would not 
prioritize changing residents’ health behaviors but 
instead emphasize building residents’ sustained power 
to advocate for equitable allocation of recreational 


esources (e.g., recreational programming, enforcement, 
and security in parks) using media, community 
engagement, and leadership-training interventions. 
These interventions could complement more traditional 
health interventions (e.g., exercise prescriptions, health 
education) and built-environment interventions (e.g., 


walking trails, playgrounds) to address the community's 
immediate and long-term recreational needs. 
Andrew M Subica, Cheryl T: Grills, Jason A. Douglas, and 


Sandra VillanuevaCommunities of Color Creating Healthy 
Environments to Combat Childhood Obesity (2016) 


26 Raimon, Martha and Kristen Weber. “Better Outcomes for Older Youth of Color in Foster Care.” https://cssp.org/resource/better-outcomes-for-older-youth-of- 


color-in-foster-care/ 


27 Pastor, Manuel et al. “Transactions, Transformations, Translations: Metrics That Matter for Building, Scaling, and Funding Social Movements” USC Program for 


Environmental and Regional Equity, 201). Available at: https://dornsife.usc.edu/assets/sites/242/docs/transactions_transformations_translations_web.pdf. 
28 Pastor, Manuel and Jennifer Ito. “Sustaining People Power.” USC Program for Environmental and Regional Equity, 2018. Available at: httos://dornsife.usc.edu/ 


assets/sites/242/docs/Sustaining_People_Power_BRIEF_FINAL.pdf p 9. 


22 Braverman, Paula et al. “The Social Determinants of Health: Coming of Age” Annual Review of Public Health. 32. 2011, pp 381-98. 


UNDERSTANDING #/¢ STRUCTURAL, SYSTEMIC, CULTURAL, 
and HISTORICAL FACTORS ¢/at are ROOT CAUSES OF INEQUITIES 


RWIJF, TCE, and others have implemented 
multi-year initiatives that seek to strengthen 


community organizing and other explicit 


the ability of communities and their 

residents to change the social determinants 
of health as a step toward the long-range 
goal of improving health outcomes at the 
population level. These initiatives are not 
programmatic in nature; they look more 
fundamentally at issues of community 
organization, community power, public policy 
and finance, system structure and practices, 
and in some cases the dominant cultural 
narratives that in themselves can shape the 
social determinants of health. The initiatives 
then assist communities (in greatly varying 
ways) to develop the organized capacity 

to address these forces. In both RWJIF’s 
Communities Creating Healthy Environments 
(CCHE) initiative and TCE’s Building Healthy 
Communities, this has involved strong 
emphasis on community mobilization and 


fforts to build power in communities that 
have been historically marginalized. The case 
study below, from the RWJIF CCHE initiative 
illustrates how communities are supported to 
address structural issues. 


Just as the RWIF and TCE sponsored initiatives 
address root causes directly and over an 
extended period of time, the evaluations 
and research on these initiatives must try to 
document and assess the interrelationships of 
the initiative’s action strategies and the changes 
in the structural and environmental factors that 
they seek to address. In the evaluation of TCE’s 
on-going work to advance health equity, which 
is adopting an equitable evaluation framework, 
this aim becomes a specific principle of the 
evaluations and efforts to extract learning now 
underway and will be a guiding principle for 
future such efforts. 


A CASE STUDY 


BUILDING EVIDENCE zm ADDRESSING ROOT CAUSES of LATINO CHILDHOOD OBESITY 


(Excerpted from Communities Creating Healthy Environments to Combat Obesity: Preliminary Evaluation Findings from Two Case 


Studies, E.R. Gonzdlez, S Villanueva, and C Grills, 2012) 


The root causes of obesity among children and youth in Latino communities are well documented. Latinos 
are less physically active than the general population, with disproportionate numbers living in low-income 
neighborhoods with environments that make it difficult for families to make healthy choices regarding 
exercise. These areas have few well equipped, safe parks, or other public open spaces for children to play and 
be active. Relative to the neighborhoods inhabited by the general U.S. population, these places also have a 
disproportionate number of fast food outlets and grocery and convenience stores that offer limited affordable 
and nutritious foods and provide an abundance of high fat and high sugar processed foods. 


La Union del Pueblo Entero (LUPE), in Hidalgo County, Texas developed a campaign to address this issue 

in its community that was supported by Robert Wood Johnson Foundation Communities Creating Healthy 
Environments (CCHE). LUPE’s policy focus is to make it easier for residents to burn calories via availability of 
health promoting environments, such as lighted streets and walking trails. 
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LUPE's CCHE campaign seeks to channel community development resources to colonias [Unincorporated, 
substandard community developed subdivisions]. LUPE’s CCHE policy-change goal is to ensure that there 
is “equitable allocation of Hidalgo County Community Development Block Grant (CDBG) funds to 


mote safe and healthy communities and address recreation equity through the creation of parks and 


implementation of streetlights in the colonias.” Their aim is improving the built environment and services 
across six colonias. This includes access to new, improved, and safer recreational spaces and development 
infrastructure, as well as access to public works services. In part, these efforts are an attempt to secure 

esources that the Texas Legislature gave county governments to install street lights; many colonias have 


yet 


rac 


wit 


es 


ins 


to receive any street lights. 


While there are no obesity data for South Texas, the Texas Department of Health Services consistently 
reports significantly higher rates of overweight and obese Latino children and adults compared to other 


ial and ethnic groups. Accordingly, LUPE is researching the allocation and distribution of rural (re) 


development and public works funding and strategizing how to bridge private and resident resources 


h county funding in select colonias. It is also developing, strengthening, and sustaining colonia 


committees to take on policy change. Some colonia committees are focusing on obtaining streetlights 
and others on creating parks/walking trails. According to LUPE, the lack of streetlights means children 
cannot safely go out and play and drivers cannot see clearly in the evenings. LUPE members/colonia 
esidents host monthly house meetings where members and general residents learn about and discuss 
community issues and public policies they feel should be implemented or stopped from implementation 
in colonias, as well as potential campaigns to achieve either of these objectives. LUPE organizers train 


idents with popular education and advocacy curriculum. 


LUPE was successful. It secured funding from three county commissioners who had committed to 


talling solar powered streetlights in five colonias and obtained a commitment by a chief administrator 


to a commissioner for a walking trail in one colonia. The funding stems from a federal grant and 
amendments to the county budget. 


FOCUSING on theVARIATION 
IN IMPACT OF INERVENTIONS 


Athird essential element in embedding equity 
principles in knowledge development and 
use has to do with a different way of focusing 
on the impacts of interventions—by looking 
continuously and creatively at the groups for 
whom interventions are not successful and 


adapting strategies until success Is achieved for all. 


“Wherever you are, do what you can and 
try to bake it into systems so they can’t 
change it when you leave.” 


— Jara Dean-Coffey, Luminare Group 


Too often now, system leaders, community 
leaders, and funders who are searching 

for new solutions look to evidence-based 
programs or best practices that have been 
shown to have success and are thus held 

up as the most desirable options for other 
communities. The strengths of evidence- 
based programs (EBPs) are that they have 
been demonstrated to work in some setting, 
for some populations, in some degree. But 
that is also their weakness, especially when 
viewed from an equity perspective: the strong 
emphasis on fidelity to evidence-based 
programs can inadvertently place barriers to 
innovation and adaptation that might help 
more children succeed. 


Initiatives that focus relentlessly on examining 
the reasons that a given intervention is not 
working for some people, and then adapt 
their interventions, often apply a continuous 
learning and improvement framework. 


Instead of holding interventions constant, 
they intentionally and carefully adapt them to 
changing and varying needs, problems, and 
opportunities.*° The aim is to systematically 
and frequently examine the causes of th 
observed variations in impact, collect the 
perspectives of those most affected, and make 
“disciplined adaptations” in the interventions 
to address what is occurring in order to better 
help each individual succeed. 


This commitment to continuing improvement 
is essential to achieving ever better outcomes, 
but raises problems for conventional 
evaluators who have been trained in methods 
that require keeping interventions constant 
and unchanged over time. It is our hope that 
evaluation methods are evolving sufficiently 
rapidly so that evaluators can support and not 
undermine adaptation and improvement. 


Whichever approach to quality improvement 
and careful adaptation of strategies is used, 
several common elements tend to be involved, 
as summarized by education researchers 
Jennifer O'Day and Marshall Smith: 


“A recent comprehensive review (of approaches 
to continuous improvement) identified five 
core features of quality improvement across 

a variety of approaches: 


1. It is focused on system outcomes for a 
defined population of beneficiaries—and 
on the processes that lead to those results; 


2. It uses variation in performance 
(including ‘failure’) as opportunities for 
learning and improvement; 


*° Bryk, Anthony. “Redressing Inequities: An Aspiration in Search of a Method.” Carnegie Foundation for the Advancement of Teaching, April 2017. Available at: 


https:/www.carnegiefoundation.org/resources/publications/redressing-inequities-an-aspiration-in-search-of-a-method/. 
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3. It takes a system perspective, with the 
understanding that systems are designed 
to get the results they produce, so if you 
want to change the results, you have to 
change the system; 


4. It is evidence-based, including measurement 
of not only outcomes but processes (and 
resources), and this measurement is 

mbedded in the day-to-day work of the 
system and its participants: and 


5. It involves a specific and coherent 
methodology and processes. Some 
of the more familiar methods include 

PDSA (Plan-Do-Study-Act) cycles, 

‘Six Sigma, and ‘LEAN.’ 


Two initiatives illustrate how systems built 
on the principles of continuous learning 
and improvement can work. The first, the 
Community College Pathways initiative (CCP) 
established by the Carnegie Foundation 
for the Advancement of Teaching, tackles 
the problem of high failure rates among 
community college students annually 
assigned to developmental (remedial) math 
instruction as a pre-requisite to taking 
college-level courses. Through a network set 
up across 50 community colleges, leaders 
at each college test innovations that, when 
successful, are rapidly shared with other 
institutions addressing the same challenges. 
In each case, the driving definition of the 
problem is to look at the students who are 
not succeeding, deeply understand why, and 
modify existing strategies or develop new 
ones that address the real barriers to their 
success. The results have been impressive, 
as documented in several publications.” 


A second example of applying evidence 
building to ensure that every individual 
succeeds can be seen in the Promise 
Neighborhoods initiative of the Northside 
Achievement Zone (NAZ) in Minneapolis. 
NAZ tracks closely the progress of each 
of their scholars (NAZ’'s term for each child 
and student, from infancy on), and works 
with the 30+ partners who collaborate on 
the North Minneapolis “cradle to college 
pipeline” to continually adapt strategies 
until success for all is achieved.*# 


Implementing this approach requires careful 
analysis of the variations being observed and 
then always asking, on behalf of individuals 
who are not succeeding, “What would it take 
for this person to succeed?” and for those who 
are succeeding, “What is working well here?” 
“What can we learn from this success?” It also 
requires equal discipline in using the evidence 
generated by the analysis to adapt practices, 
programs, and ultimately systems and policies 
to not just accommodate but advance the 
necessary supports and services. 


Evidence development around these 
highly-tailored approaches must focus 

on understanding and documenting how 
the process of innovation and adaptation 
moves forward (since that is a vital part of 
“what's working”), as well as documenting 
the combination of programmatic and other 
supports that help each child, youth, or 
family succeed. 


3! O'Day, Jennifer A. and Marshall S. Smith. (2016) “Equality and Quality in U.S Education: Systemic Problems, Systemic Solutions.” The Dynamics of Opportunity 


in America, September 2016,) p. 315. 


32 Lachman, Sherry. “Carnegie Math Pathways: Case Study.” Center for the Study of Social Policy, 2016. Available at: https://cssp.org/resource/carne- 


gie-math-pathways/ 


33 Aberdin, Shanti. “Northside Achievement Zone: Case Study Center for the Study of Social Policy, 2016. Available at: https://cssp.org/wp-content/up- 


loads/2018/08/Northside-Achievement-Zone.pdf. 
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Strategies to advance equity must be 
accompanied by new ways of generating 

and using evidence. In this paper, we have 
highlighted thr lements that do not 

stand alone but are facets of an overall 
commitment to equity in how evidence 

is developed and used. By (1) engaging in 
knowledge develooment/evidence generation 
in genuine partnership with those whose lives 
are most affected, (2) taking into account root 
causes of inequities and developing sound 
measures for addressing and tracking their 
continued influence over equitable outcomes, 
and (3) focusing on continuous learning and 
improvement to inform disciplined adaptation 
of strategies regularly and systematically to 
address the needs of those for whom they are 
not yet effective, the process of developing and 
using evidence can advance the achievement 
of more equitable outcomes at scale. 


For these practices to become the norm in 
knowledge development about complex 
strategies, however, much has to change. 


For these practices to become the norm in 
knowledge development about complex 
strategies, however, much has to change. 

The ways in which research and evaluation 
activities are conceptualized, funded, and 
positioned within the broader world of social 
change have to be altered. This will require change 
for communities, for funders, and for evaluators. 


For communities, several changes 
seem essential: 


Resourcing communities to be 
partners in evaluation and knowledge 
development. We have highlighted 
methodologies in which the balance of 


power is shifted so that community 
members are co-equals and partners 
in generating knowledge. Yet their 
contributions are rarely compensated. 
For equity in knowledge development 
at all stages—in design, in conduct, 

in develooment of findings— 
community members’ time should 

be financially supported. 

Developing community members’ 
capacity to participate in, “own,” 

and lead evaluation activities. As 
community members are recognized 
for the expertise they contribute 

to knowledge development, they 

are seen not just as sources of 
information but as active participants 
in, and in some cases, leaders of 
knowledge development. This requires 
capacity building, however. The 

skills for observing, documenting, 

and synthesizing experienc 

can be expanded and honed, 

and as community participatory 
methodologies are used mor 
frequently, systematic opportunities for 
community members to acquire these 
skills should be created. 

Broader use of results-based, 
continuous improvement approaches 
to learn from and guide complex 
community initiatives. The Northside 
Achievement Zone (NAZ) exampl 
illustrates the power of a rigorous 

and on-going approach to reviewing 
data and experience in order to guide 
change. NAZ’s collection, analysis, and 
use of data are deeply rooted in what 
parents and community members 
want to understand about their 
children’s development and educational 
experience and success. The process 
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of reflecting on progress, adjusting 
strategies, and affirming commitment 
to ambitious goals does not have to 
be foreign when it is embedded in 
communities, with leadership roles for 
community members. These processes 
cost—but resources used in this way 
pay off in the results they achieve. 

As initiatives like NAZ systematically 
collect, analyze, and use this broader 
range of data and experience, w 
expect that increasingly this data 

and experience will become part of 

a broader knowledge base that will 
inform and guide equally complex 
emerging initiatives. 


Advancing equity in the develooment and use 
of evidence has implications for funders as well: 


Funders should consider adopting 

an overall framework that unites the 
concepts of equity and evaluation and 
that constitutes a way of viewing the 
knowledge development enterprise. 
The Equity Evaluation Initiative (EEl) is 
developing and testing such a framework 
in partnership with a number of 
philanthropic funders.* The developers 
view the process of using the EE! 
framework a learning process in its own 
right, as they believe its full value will 

be discovered by the foundations and 
evaluators who are now implementing it 
and enriching it in theory and practice. 
Whether using the EE! framework or a 
variation or alternative to it, incorporating 
strong and concrete guiding principles 
that link equity and the generation and 
use of evidence seems essential. 


- Within an overall equity framework, 


funders should focus more on 
understanding and documenting the 
impact of root causes and systemic 
and structural racism and oppression, 
as these are the greatest contributors 
to why disparities persist. 

This recommendation applies not just to 
the evaluation and evidence-gathering 
roles of funders, of course, but to the 
initiatives and strategic approaches they 
support to address disparities. Both in 
what is done and what is measured, 
more attention must be paid to historical 
forces and underlying causes linked to 
perpetuating unequal outcomes. 
Funders must be prepared to share 
power in the generation and use of 
evidence. Leaders in the foundation field 
are increasingly aware of the need for 
humility and power-sharing as they enter 
into relationship with communities, even 
if there is considerable progress still to 
be made in translating this awareness 
into practice. However, the approaches 
discussed in this paper and advocated in 
efforts such as EEI require equal humility 
and power-sharing with community 
partners in the conduct of research and 
evaluation. As new approaches take hold, 
funders must see themselves more as 
supporters, stewards, and contributors 
to knowledge development, not as the 
owners or determiners of knowledge. 
They are lifting up and adding value to 
the knowledge gained from experiences 
that are not ultimately theirs, and this 
requires a stance of humility, partnership, 
anda willingness to share power and be 
aware of their considerable privilege. 


** Center for Evaluation Innovation, Institute for Foundation and Donor Learning, Dorothy A Johnson Center for Philanthropy and Luminare Group. “Equitable 
Evaluation Framing Paper.” July 2017. Available at: https://drive.google.com/file/d/IRBWoWVduMvh4S68qdmPLN3WPEyPWOwOS/view. 
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Funders may need to reconsider what 
they regard as the most valuable 
knowledge. As understanding grows 
that the complex, fundamental causes 
of disparities are only lightly addressed 
(if at all) by programs and projects, 

the recognition of the limits of linear, 
simplified notions of evidence change. 
The recorded knowledge journey 

of the Lankelly Chase Foundation 

in Great Britain captures this view: 
“People we call ‘learning partners’ 
(note: not evaluators) attached to the 
projects we are funding are actually 
an embodiment of the future we are 
seeking. They represent a rejection 

of a paradigm where knowledge is 
generated to service a machine that 
wants certainty where certainty is 

not possible. Instead, they are using 
knowledge as a reflective tool to help 
those doing the work to navigate their 
way. In some cases the organizations 
we are working with have rejected the 
machine entirely and will not collect 
what they consider to be meaningless 
data. They know it is actually impossible 
to prove their work has a direct linear 
effect on any particular outcome of 
interest to funders or public bodies— 
they won't get involved in that dance 
of deceit. They realize that even having 
outcomes or targets in place risks 
skewing the relational, adaptive, 
human way they work.”*° 


Shifting the relationship between funder, 
evaluator, and evaluation participants to 
be more equitable, and to create evaluative 
processes that imbue social change agents 
with data and value their insights which 
inform their own strategies (and that of the 
foundation), requires a nonconventional 
evaluator (compared to the larger field) 
with specialized training, an explicit values 
framework, and a different approach to 
evaluation design and management. 


EQUITABLE EVALUATION FRAMEWORK 


Finally, the approaches discussed here mean 
many changes for evaluators. 


Evaluators must be prepared to be 
advocates not bystanders. Researchers 
and evaluators need to stand on the side 
of communities of color, be vocal advocates 
that communities need a minimum of ten, 
if not 25 years or more of deep investment 
and evaluation and learning to turn the 
curve on social inequity. 

Evaluators should draw on the growing 
body of work and available resources 
dedicated to increasing the equity of 
knowledge generation. These resources 
include the principles, frameworks, 

and practices of Community-Based 
Participatory Research, the Indigenous 
Evaluation Framework, the principles of 
the Equitable Evaluation Initiative, and 
the tools found at Racial Equity Tools. 
This guidance from multiple sources 


38 “Our Knowledge Journey So Far.” Lankelly Chase, June 21, 2018. Available at: https://lankellychase.org.uk/our-work/knowledge/learning/. 
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has a consistent message about how 
to proceed: authentic partnerships, 
alignment of design with community 
values and priorities, and rigorous but 
flexible methods. 

Evaluators should adhere to equitable 
evaluation principles. The Equitable 
Evaluation Framework specifies three 


evaluation principles. Evaluations should: 


Be in service offcontribute to equity. 
Result in information that describes 
effects of strategy on different 
populations, underlying systemic 


drivers, and history and cultural context. 


Designed and implemented with 
cultural competency, multi-cultural 
validity, and an orientation toward 
participant ownership. 

Evaluators should work with 

communities and funders to develop 

standards. The framers of Equitable 

Evaluation have noted, “There is not 

yet a consistent understanding in the 

field of common standards of practice 

around what evaluation practice of 
equity efforts—and equitable evaluation 
more broadly—should look like.”%6 The US 


government and academic institutions 
have established human subject research 
protection protocols as an ethical baseline 
for how individuals are included in 
research. The communities, researchers, 
and evaluators need a set of standards 
around how to engage community 
members to ensure meaningful 
participation. These standards can 

be drawn from and build on the 

work of that frames the Indigenous 
Evaluation Framework and should create 
expectations for empowerment and 
protection of those most affected by 
interventions in the evidence enterprise 
that surrounds them. 

Evaluators should seek to enhance 
their skills knowledge of methods. 
Evaluation is more than a technical skill. 
t requires competencies in convening, 
facilitating, engaging and coaching 
diverse teams and communities to build 
evidence that advances equity. Likewise, 
evaluators should bring an entire 

menu of methods to evaluation and be 
flexible in their implementation while 
maintaining rigor. 


6 Center for Evaluation Innovation, Institute for Foundation and Donor Learning, Dorothy A Johnson Center for Philanthropy and Luminare Group. “Equitable 
Evaluation Framing Paper.” July 2017. Available at: https://drive.qgoogle.com/file/d/IRBWoWVduMvh4S68qdmPLN3WPEyPWOwOS/view. 
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ONCLUSION 


In considering “putting equity at the center,” 
one point emerges as the most fundamental. 
The separation often made between th 
strategies to advance equity and the 
development and use of knowledge about 
these activities are not useful. These should 
not be seen as separate and disconnected 
tracks. The principles that guide action must 
guide evidence development as well, for both 


and it is only as this aim becomes reality that 
the dual goals—achieving equity and knowing 
more about how equity can be achieved 
and sustained—will be accomplished. 


to be successful. Equity is at the center of both, 


N 


